由于COVID-19，许多学校通过视频会议软件在线考试已经采用了许多学校。虽然方便，但教师要同时显示的学生变焦窗口监督在线考试是具有挑战性的。在本文中，我们提出了IEXAM，这是一种智能的在线考试监测和分析系统，不仅可以使用面部检测来帮助监护人实时学生识别，而且还可以检测到常见的异常行为（包括面部消失，旋转的面部，旋转的面部，旋转，，旋转，并在考试期间用另一个人替换）通过基于面部识别后的外观后视频分析。为了建立这样的新型系统，我们克服了三个挑战。首先，我们发现了一种轻巧的方法来捕获考试视频流并实时分析它们。其次，我们利用每个学生的变焦窗口上显示的左角名称，并提出了改进的OCR（光学角色识别）技术来自动收集具有动态位置的学生面孔的地面真相。第三，我们进行了几次实验比较和优化，以有效缩短教师PC所需的训练时间和测试时间。我们的评估表明，IEXAM可以实现高精度，实时面部检测为90.4％，后验后面部识别率为98.4％，同时保持可接受的运行时性能。我们已经在https://github.com/vprlab/iexam上提供了IEXAM的源代码。

使用和部署不同本地模型的个性化联合学习（PFL），由于其在处理佛罗里达州客户的统计异质性方面的成功，近年来引起了人们的关注。但是，对不同PFL方法的标准化评估和系统分析仍然是一个挑战。首先，高度多样化的数据集，FL仿真设置和PFL实现可以防止对PFL方法的快速和公平比较。其次，在各种实践场景中，PFL方法的有效性和鲁棒性不足，例如新客户的概括和资源有限的客户参与。最后，当前的PFL文献在采用的评估和消融方案中有所不同。为了应对这些挑战，我们提出了第一个全面的PFL基准PFL基准，以促进快速，可重现，标准化和彻底的PFL评估。所提出的基准测试包含具有统一数据分区和现实异质设置的不同应用程序域中的10多个数据集；一个模块化且易于扩展的PFL代码库，具有20多个竞争性PFL基线实现；以及在集装环境下进行的系统评估，以概括，公平，系统开销和收敛性。我们强调了最先进的PFL方法的好处和潜力，并希望PFL板台实现了进一步的PFL研究和广泛的应用，否则由于缺乏专用的基准，这将是困难的。该代码在https://github.com/alibaba/federatedscope/tree/master/master/benchmark/pfl-bench上发布。

为了调查现实世界中联邦学习的异质性，我们将经典的联合学习概括为联合的异性任务学习，这强调了参与者在数据分布和学习任务方面的联盟学习中的不一致性。我们还提出了B-FHTL，这是一种联合的杂项任务学习基准，该基准包括模拟数据集，FL协议和统一的评估机制。 B-FHTL数据集包含三个精心设计的联合学习任务，异质性增加。每个任务都使用不同的非IID数据和学习任务模拟客户端。为了确保不同的FL算法之间的公平比较，B-FHTL通过提供高级API来避免隐私泄漏，在整个FL协议中构建，并预设跨越不同的学习任务的最常见评估指标，例如回归，分类，文本，文本，文本此外，我们还比较了B-FHTL中联合多任务学习，联合个性化和联合元学习领域的FL算法，并突出了联盟异质任务学习的异质性和困难的影响。我们的基准测试，包括联合数据集，协议，评估机制和初步实验，可在https://github.com/alibaba/federatedscope/tree/master/master/master/benchmark/b-fhtl上开放。

尽管现有联合学习平台（FL）平台已取得了显着的进展，以提供开发基础架构，但这些平台可能无法很好地应对各种异质性带来的挑战，包括参与者本地数据，资源，行为和学习目标中的异质性。为了填补这一空白，在本文中，我们提出了一个名为FederatedScope的新型FL平台，该平台采用事件驱动的架构为用户提供极大的灵活性，以独立描述不同参与者的行为。这样的设计使用户可以轻松地描述参与者具有各种本地培训过程，学习目标和后端，并通过同步或异步培训策略将其协调为FL课程。 FederatedScope为易于使用和灵活的平台提供了丰富类型的插入操作和组件，以有效地进行进一步开发，并且我们实施了几个重要组件，以更好地帮助用户进行隐私保护，攻击模拟和自动调整。我们已经在https://github.com/alibaba/federatedscope上发布了FederatedScope，以在各种情况下促进联邦学习的学术研究和工业部署。

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

Weakly-supervised object localization aims to indicate the category as well as the scope of an object in an image given only the image-level labels. Most of the existing works are based on Class Activation Mapping (CAM) and endeavor to enlarge the discriminative area inside the activation map to perceive the whole object, yet ignore the co-occurrence confounder of the object and context (e.g., fish and water), which makes the model inspection hard to distinguish object boundaries. Besides, the use of CAM also brings a dilemma problem that the classification and localization always suffer from a performance gap and can not reach their highest accuracy simultaneously. In this paper, we propose a casual knowledge distillation method, dubbed KD-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention (CI), which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the de-biased object feature, we additionally propose a multi-teacher causal distillation framework to balance the absorption of classification knowledge and localization knowledge during model training. Extensive experiments on several benchmarks demonstrate the effectiveness of KD-CI-CAM in learning clear object boundaries from confounding contexts and addressing the dilemma problem between classification and localization performance.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.

In this work, we focus on instance-level open vocabulary segmentation, intending to expand a segmenter for instance-wise novel categories without mask annotations. We investigate a simple yet effective framework with the help of image captions, focusing on exploiting thousands of object nouns in captions to discover instances of novel classes. Rather than adopting pretrained caption models or using massive caption datasets with complex pipelines, we propose an end-to-end solution from two aspects: caption grounding and caption generation. In particular, we devise a joint Caption Grounding and Generation (CGG) framework based on a Mask Transformer baseline. The framework has a novel grounding loss that performs explicit and implicit multi-modal feature alignments. We further design a lightweight caption generation head to allow for additional caption supervision. We find that grounding and generation complement each other, significantly enhancing the segmentation performance for novel categories. We conduct extensive experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS). The results demonstrate the superiority of our CGG framework over previous OVIS methods, achieving a large improvement of 6.8% mAP on novel classes without extra caption data. Our method also achieves over 15% PQ improvements for novel classes on the OSPS benchmark under various settings.

Nearest-Neighbor (NN) classification has been proven as a simple and effective approach for few-shot learning. The query data can be classified efficiently by finding the nearest support class based on features extracted by pretrained deep models. However, NN-based methods are sensitive to the data distribution and may produce false prediction if the samples in the support set happen to lie around the distribution boundary of different classes. To solve this issue, we present P3DC-Shot, an improved nearest-neighbor based few-shot classification method empowered by prior-driven data calibration. Inspired by the distribution calibration technique which utilizes the distribution or statistics of the base classes to calibrate the data for few-shot tasks, we propose a novel discrete data calibration operation which is more suitable for NN-based few-shot classification. Specifically, we treat the prototypes representing each base class as priors and calibrate each support data based on its similarity to different base prototypes. Then, we perform NN classification using these discretely calibrated support data. Results from extensive experiments on various datasets show our efficient non-learning based method can outperform or at least comparable to SOTA methods which need additional learning steps.

In this tutorial paper, we look into the evolution and prospect of network architecture and propose a novel conceptual architecture for the 6th generation (6G) networks. The proposed architecture has two key elements, i.e., holistic network virtualization and pervasive artificial intelligence (AI). The holistic network virtualization consists of network slicing and digital twin, from the aspects of service provision and service demand, respectively, to incorporate service-centric and user-centric networking. The pervasive network intelligence integrates AI into future networks from the perspectives of networking for AI and AI for networking, respectively. Building on holistic network virtualization and pervasive network intelligence, the proposed architecture can facilitate three types of interplay, i.e., the interplay between digital twin and network slicing paradigms, between model-driven and data-driven methods for network management, and between virtualization and AI, to maximize the flexibility, scalability, adaptivity, and intelligence for 6G networks. We also identify challenges and open issues related to the proposed architecture. By providing our vision, we aim to inspire further discussions and developments on the potential architecture of 6G.